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I. 


INTRODUCTION 


Unmanned  vehicles  have  emerged  as  the  next  wave  of  technology  to  populate  the 
battlefield.  Whether  this  technology  will  dominate  the  battlespace  is  still  uncertain,  but 
one  thing  is  certain:  unmanned  vehicles  have  already  changed  how  we  operate. 
Unmanned  systems  have  allowed  access  to  previously  inaccessible  areas,  increased 
mission  efficiency,  and  extended  the  reach  of  a  host  platform.  They  provide  valuable 
time- sensitive  information.  The  U.S.  Navy  defines  unmanned  undersea  vehicles  (UUVs) 
as  “self-propelled  submersibles  whose  operation  is  either  fully  autonomous  or  under 
minimal  supervisory  control  and  is  untethered  except,  possibly,  for  data  links  such  as  a 
fiber  optic  cable”  [1].  This  definition  is  not  meant  to  encompass  Remote  Operated 
Vehicles  (ROVs)  [1]. 

During  Operation  Iraqi  Freedom,  UUVs  were  used  to  clear  mines  in  the  Persian 
Gulf  [1].  Other  UUVs  have  been  used  to  conduct  oceanographic  surveys  or  underwater 
inspections.  The  Navy  is  focusing  on  nine  missions  according  to  the  UUV  master  plan: 
“Intelligence,  Surveillance,  and  Reconnaissance;  Mine  Countermeasures;  Anti- 
Submarine  Warfare;  Inspection  /  Identification;  Oceanography;  Communication  / 
Navigation  Network  Nodes;  Payload  Delivery;  Information  Operations;  and  Time 
Critical  Strike”  [1].  The  UUV  master  plan  also  states  that  unmanned  vehicles  will  serve 
as  force  multipliers  and  risk  reduction  agents  in  these  missions.  The  long-term  UUV 
vision  is  “to  have  the  capability  to:  (1)  deploy  or  retrieve  devices;  (2)  gather,  transmit,  or 
act  on  all  types  of  information;  and  (3)  engage  bottom,  volume,  surface,  air,  or  land 
targets”  [1]. 

UUVs  must  be  autonomous,  deployable,  adaptable,  persistent,  and  low  profile  to 
contribute  to  the  needs  of  the  Navy  in  accomplishing  the  intended  missions.  As  set  forth 
in  the  UUV  master  plan  [1],  autonomy  is  the  ability  to  operate  independently  for 
extended  periods  of  time.  It  also  describes  deployable  as  meaning  that  the  UUV  can  be 
easily  deployed  and  recovered  from  a  variety  of  platforms  and  in  large  quantities. 
Persistence  refers  to  a  vehicle’s  ability  to  stay  on  station  for  long  periods  of  time, 

including  in  adverse  weather  conditions  as  described  by  the  UUV  master  plan.  Low 
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profile  means  that  the  vehicle  will  operate  covertly  with  low  acoustic  and  electromagnetic 
signature  [1].  Autonomy  is  considered  the  largest  long-term  challenge  for  UUVs, 
followed  closely  by  the  power  and  energy  technology,  according  to  a  study  performed  by 
the  RAND  Corporation  [2]. 

The  Office  of  Naval  Research  (ONR)  has  defined  six  levels  of  vehicle  autonomy: 

1)  Fully  Autonomous  defines  a  system  that  “requires  no  human  intervention 
to  perform  any  of  its  designed  activities”  [3]. 

2)  Mixed  Initiative  describes  an  autonomous  system  that  allows  either  the 
system  or  a  human  to  react  to  sensed  data.  “The  system  can  coordinate  its 
behavior  with  human  behavior  both  explicitly  and  implicitly”  [3]. 

3)  A  Human  Supervised  system  “can  perform  a  wide  variety  of  activities 
once  given  top-level  permission  or  direction  by  a  human”  [3]. 

4)  Human  Delegated  systems  “can  perform  limited  control  activities  on  a 
delegated  basis”  [3]. 

5)  Human  Assisted  systems  “can  perform  activities  in  parallel  with  human 
input. . .  However,  the  system  has  no  ability  to  act  without  accompanying 
human  input”  [3]. 

6)  Human  Operated  defines  a  system  with  no  autonomy  [3]. 

For  the  purpose  of  this  research.  Autonomous  Underwater  Vehicles  (AUVs)  will 
include  vehicles  described  by  the  third  and  fourth  categories,  and  Remotely  Operated 
Vehicles  (ROVs)  will  include  vehicles  from  the  fifth  and  sixth  categories.  There  are  not 
currently  any  underwater  vehicles  that  would  be  classified  in  the  first  two  categories. 

ROVs  require  a  full-time  remote  operator,  a  tether  for  power  and  data  transfer, 
and  a  camera  for  the  operator  to  see  the  environment.  AUVs  are  generally  programmed  to 
swim  a  specific  path  and  do  not  require  an  operator  or  a  tether.  Tethered  vehicles  restrict 
the  host  platform,  create  entanglement  scenarios,  may  be  difficult  to  recover  quickly  and 
are,  therefore,  not  suitable  for  the  battlefield  except  in  very  specific  circumstances  [1].  A 
tether  is  helpful  during  research  and  development  as  UUVs  progress  to  fulfill  the 
intended  missions.  Current  technology  and  design  are  not  yet  sufficient  for  autonomous 
vehicles  to  accomplish  most  of  the  specified  missions  [4]. 
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Increased  autonomy  is  required  to  conduct  the  desired  missions  [1].  To  achieve 
the  required  level  of  autonomy,  vehicles  must  be  able  to  sense,  process,  and  react  to  both 
static  and  dynamic  environments.  Robot  vision  is  defined  as  the  combination  of  hardware 
and  software  algorithms  to  allow  a  robot  to  process  its  environment  [5].  Robot  vision 
must  be  developed  first  to  sense  and  process  the  environment  before  controllers  can  be 
developed  to  employ  these  techniques. 

Sensors  currently  available  include  various  sonar  systems,  laser  rangefinders, 
structured  light,  and  visual  odometry.  Of  these  many  sensors,  none  are  ready  for 
development  into  controllers  for  an  AUV  to  rapidly  adapt  to  a  changing  environment  [2] ; 
although,  there  is  a  lot  of  research  currently  underway  to  move  in  that  direction.  With 
current  technology,  either  sensing  the  environment  takes  too  long  or  processing  the  data 
is  beyond  the  onboard  capability;  therefore,  developing  a  successful  form  of  robot  vision 
for  AUVs  is  a  worthwhile  field  of  study. 

A.  RESEARCH  OBJECTIVE 

The  purpose  of  this  research  is  to  demonstrate  a  new  technique  for  underwater 
robot  vision  using  image  analysis  that  may  be  developed  into  a  controller  for  AUVs. 
Analyzing  the  change  in  wavelength-specific  light  intensity  as  it  travels  through  water 
from  an  object  to  a  camera  may  be  used  to  approximate  the  distance  between  the  object 
and  the  camera.  If  the  intensities  cannot  be  measured,  then  a  relative  distance  can  be 
determined  by  comparing  the  color  shift  that  light  experiences  between  the  object  and  the 
camera.  This  technique  uses  a  camera,  a  filter,  and  relies  on  the  existence  of  natural  light. 

B.  THESIS  LAYOUT 

This  thesis  begins  with  a  description  of  robot  vision  and  a  brief  discussion  of 
advantages  or  obstacles  for  adapting  these  sensors  in  Chapter  II.  The  development  of  a 
new  technique  for  underwater  robot  vision  using  a  color  shift  is  described  in  Chapter  III. 
The  equipment  and  design  used  to  collect  and  analyze  data  to  prove  the  technique  is 
included  in  Chapter  IV.  The  results  are  summarized,  including  conclusions  and 
recommendations  for  future  work,  in  Chapter  V. 
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II.  BACKGROUND 


A.  ROBOT  VISION 

Current  sensors  are  insufficient  to  develop  into  useful  underwater  robot  vision. 
The  time  required  to  gather  and  process  the  data  takes  too  long.  Three  knots  is  considered 
a  low  speed  for  an  AUV.  At  this  speed  the  AUV  is  moving  1.54  meters  per  second  (5.05 
feet  per  second).  The  environment  needs  to  be  sensed,  interpreted,  and  a  course 
correction  made  within  a  fraction  of  this  time  to  avoid  obstacles. 

Most  AUVs  are  programmed  to  swim  a  specific  pattern  and  area  without  being 
able  to  “see”  where  it  is  going  at  all.  The  AUV  will  use  onboard  sensors  such  as  an  echo 
sounder.  Global  Positioning  System,  Inertial  Measurement  Unit,  or  Doppler  Velocity  Log 
to  verify  its  position  and  continue  along  its  preprogrammed  path.  If  there  happens  to  be  a 
new  object  in  its  path  or  if  it  deviates  from  its  programmed  course  it  can  get  lost  or  stuck. 

There  are  currently  many  sensors  and  techniques  under  development  for 
underwater  robot  vision.  There  are  laser-based  sensors,  which  project  a  laser  and 
calculate  ranges  based  on  time-of-flight  calculations  while  making  some  assumptions 
about  the  scene  geometry  [6].  One  method  developed  by  Cain  and  Leonessa  [6]  and 
further  work  by  Hansen  et  al.  [7]  utilizes  two  line  lasers  and  a  camera  to  provide  a  two- 
dimensional  and  three-dimensional  representation  of  the  environment,  respectively. 
Several  other  laser-based  systems  have  been  developed  by  researchers  such  as  Karras  and 
Kyriakopoulos  [8],  Jaffe  [9],  and  Campos  and  Godina  [10].  The  approach  by  Campos  and 
Godina  bridges  the  next  subject  in  that  it  uses  lasers  to  project  structured  light  patterns  to 
determine  the  shape  of  objects  in  the  environment. 

Structured  light  is  a  technique  that  has  recently  received  some  attention. 
Structured  light  works  like  laser  scanners  by  projecting  light  and  then  viewing  the 
reflected  light  with  a  camera  set  at  an  angle.  The  difference  is  that  the  light  projected  has 
a  specific  pattern  instead  of  just  a  point  or  beam.  Comparing  the  expected  pattern 
(assuming  no  object  in  the  path  of  the  light)  to  the  actual  return  can  determine  the  shape 
of  the  object  that  caused  the  distortion.  The  projected  light  may  be  black  and  white. 
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colored  or  even  at  higher  frequencies  such  as  infrared  or  ultraviolet  and  may  be  projected 
in  an  infinite  variety  of  patterns  [10]-[14].  Other  variations  may  include  two  or  more 
cameras  at  various  angles  to  improve  accuracy  or  to  compensate  for  the  directionality  of 
the  pattern  [15]-[17].  Different  patterns  may  be  projected  sequentially  and  then  stitched 
together  to  form  a  point  cloud.  The  resolution  of  the  resultant  point  cloud  is  limited  by 
the  resolution  and  complexity  of  the  projected  pattern. 

Another  method  of  robot  vision  is  based  on  a  technique  called  visual  odometry. 
“Visual  odometry  is  the  process  of  determining  the  position  and  orientation  of  a  robot  by 
analyzing  associated  camera  images”  [18].  Images  are  acquired  using  either  a  single 
camera  or  multiple  cameras  working  in  stereo  or  omnidirectional  cameras  [19]-[24]. 
Visual  odometry  can  be  done  at  a  fraction  of  the  cost  and  computing  power  of  other  robot 
vision  methods  [19].  It  has  been  studied  for  decades  and  may  be  the  answer  to  autonomy 
in  underwater  vehicles. 

B.  THE  BEER-LAMBERT  LAW 

The  way  light  interacts  with  the  ocean  is  peculiar  and  has  been  studied  for 
decades.  Light  changes  as  it  enters  the  water  and  as  it  travels  to  the  depths  it  continues  to 
change.  Perception  of  light  underwater  changes  as  well.  Because  of  refraction,  objects 
appear  larger  underwater  unless  viewed  through  a  domed  lens.  Colors  are  difficult  to  see. 
Most  things  appear  as  a  shade  of  green  and  the  deeper  one  goes  the  darker  and  colder 
everything  appears. 

Most  light  that  reaches  the  ocean  surface  is  transmitted  into  the  sea  and  attenuated 
in  the  water  below.  Attenuation  is  due  to  absorption  and  scattering  as  discussed  by 
Kirk  [25].  Absorption  is  of  particular  interest  because  light  at  different  wavelengths 
experiences  higher  or  lower  absorption  over  the  same  distance.  Red  light  is  absorbed  over 
a  short  distance  and  may  only  travel  up  to  10  m  through  clear  salt  water,  whereas  green 
light  may  travel  up  to  25  times  as  far  before  it  is  absorbed,  as  seen  in  Figure  1  [26].  Light 
travels  significantly  less  distance  in  coastal  waters  as  seen  in  the  right-hand  side  of 
Figure  1.  This  is  primarily  due  to  scattering.  More  particulate  in  the  water  causes  the  light 
to  scatter  before  it  can  be  absorbed.  Some  regions  have  such  high  concentrations  of 
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particulate  that  none  of  the  wavelengths  travel  more  than  a  few  inehes;  in  other  words, 
nothing  is  visible  even  inehes  away. 

Light  penetration  in  open  ocean  Light  penetration  in  coastal  waters. 

Depth  in 
meters 

50 

100 

150 

200 

Figure  1 .  Patterns  of  Light  Penetration  in  Open  Water  and  Coastal  Water. 

Souree:  [27]. 

Underwater  photography  and  videography  requires  additional  light  sourees  or 
filters  to  restore  visible  wavelengths  of  light  to  eompensate  for  this  absorption  [28].  This 
absorption  is  predietable  and  is  used  to  develop  robot  vision.  The  absorption  of  light  in 
water  may  be  deseribed  by  the  Beer-Lambert  law  [29], 

h  =  (2.1) 

where  Id  represents  the  intensity  of  the  light  at  a  given  distance  d  and  lo  represents  the 
intensity  of  the  light  at  the  souree.  The  absorption  eoefficient  is  represented  by  a.  It  is 
easy  to  see  that  this  represents  an  exponential  deeay  proportional  to  the  distance  and 
absorption  eoefficient  for  a  given  wavelength.  Many  studies  have  been  done  to  determine 
the  speeifie  absorption  eoeffieients  at  various  wavelengths  in  pure  water  as  well  as  other 
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fluids  to  improve  our  understanding  of  the  relationship  between  light  and  water 
[30]-[32]. 

C.  COLOR  SPACE 

RGB  values  are  related  to  three  standard  primaries  ealled  X,  Y,  and  Z  by  the 
International  Commission  on  Illumination  or  Commission  Internationale  de  I’Eclairage 
(CIE)  as  early  as  1931  [33],  [34].  The  XYZ  eolor  spaee  is  an  international  standard  used 
to  define  colors  that  is  invariant  across  devices  and  is  seen  in  Figure  2.  These  primaries 
are  then  correlated  to  specific  wavelengths  of  light.  This  comparison  links  the  physical 
pure  colors  to  the  physiological  perceived  colors  and  defines  the  XYZ  color  space  and  the 
RGB  color  space  [34],  [35].  The  RGB  color  space  varies  between  devices  as  a  local 
device’s  interpretation  of  the  XYZ  color  space  standard.  The  Adobe  RGB  primary 
triangle  is  also  shown  in  Figure  2.  It  is  noted  that  colors  outside  this  triangle  cannot  be 
represented  when  adopting  the  Adobe  RGB  standard.  The  Adobe  RGB  color  space  is 
only  able  to  represent  about  50  percent  of  the  XYZ  color  space  but  is  the  best 
representation  currently  available. 


Figure  2.  CIE  XYZ  Color  Space  Including  Adobe  RGB  Triangle  and  D65  White 

Standard.  Source:  [36]. 
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The  (x,y)  values  in  the  triangle  in  Figure  2  are  correlated  with  corresponding  RGB 
values  between  zero  and  one.  Corners  correspond  to  the  points  in  Table  1  [36].  Each  red, 
green,  and  blue  value  that  makeup  a  color  is  typically  stored  as  an  8-bit  byte  for  most 
devices,  although  higher  resolution  is  available  on  some  devices.  A  one  corresponds  to 
255,  and  each  corner  is  represented  as  (255,0,0)  “red,”  (0,255,0)  “green,”  and  (0,0,255) 
“blue.”  For  every  fraction  of  each  of  these  values  there  is  a  corresponding  wavelength  of 
color.  For  example,  a  wavelength  of  620  nm  corresponds  to  an  RGB  value  of  (255,0,0)  or 
(1,0,0),  the  brightest  red.  “Brightest”  may  be  misleading  and  refers  to  the  shade  of  red 
and  not  the  typical  brightness.  The  combination  of  RGB  values  indicates  the  color  of  a 
pixel  but  is  independent  of  intensity. 

Table  1.  Red,  Green,  Blue  Primaries  in  Adobe  RGB  Color  Space  Correlated 

to  (x,y)  Values.  Adapted  from  [36]. 


X 

y 

R 

G 

B 

Red 

0.64 

0.33 

1 

0 

0 

Green 

0.21 

0.71 

0 

1 

0 

Blue 

0.15 

0.06 

0 

0 

1 

White 

0.3127 

0.329 

1 

1 

1 

D.  LUMINOUS  INTENSITY 

The  intensities  described  in  Equation  (2.1)  are  luminous  intensities  and  are 
measured  in  candela  or  lumen  per  steradian.  Fuminous  intensity  is  the  wavelength- 
weighted  quantity  of  visible  light  that  is  emitted  per  unit  time  per  solid  angle  [37].  By 
definition,  “if  a  light  source  emits  monochromatic  green  light  with  a  wavelength  of  555 
nm  and  has  a  radiant  intensity  of  1/683  watts  per  steradian  in  a  given  direction,  that  light 
source  will  emit  one  candela  in  the  specified  direction”  [38].  A  submersible  radiometer 
has  been  designed  by  Morrow  et  al.  [39]  to  “measure  ultraviolet  and  visible  light  and 

detect  extremely  low  light  levels  in  the  deep  sea.  They  numerically  describe  the  shape  of 
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the  light  field  in  the  ocean  and  measure  how  light  is  absorbed  and  scattered  in  water  over 
small  spatial  scales”  [39].  These  radiometers  may  be  used  to  precisely  measure  the 
intensity  of  various  wavelengths  or,  more  importantly,  the  change  in  the  intensity  of  a 
specific  wavelength  over  a  given  distance. 

E.  CORRECTING  FOR  TEMPERATURE  AND  SALINITY 

Pinpointing  the  specific  absorption  coefficient  proves  difficult  as  it  varies  with 
temperature,  salinity,  and  turbidity.  It  may  vary  for  the  same  location  because  of  seasonal 
changes  in  temperature  and  salinity  and  even  over  short  periods  of  time  based  on  a  local 
event.  Even  in  a  spot  in  the  middle  of  the  ocean  where  the  temperature  and  salinity  have 
little  seasonal  variation  and  the  water  is  very  clear,  there  are  thermal  layers  and  pockets 
that  do  not  mix  well  throw  off  this  calculation.  The  chosen  wavelength  of  light  to  use  can 
improve  this  estimate;  likewise  comparing  between  several  wavelengths  may  also 
mitigate  these  inaccuracies. 

The  absorption  coefficient  can  be  corrected  for  temperature  and  salinity  [40]. 
These  coefficients  have  been  spectrophotometrically  measured  and  verified  using  a  point- 
source  integrating  cavity  absorption  meter  [41].  Shorter  wavelengths  have  a  positive 
correlation  with  temperature,  and  longer  wavelengths  have  a  negative  correlation  with 
temperature  centered  around  650  nm  [41].  These  corrections  are  made  by  adding  the 
temperature  dependence  Wt  and  the  salinity  dependence  Ws  to  the  absorption  coefficient 
a  [40] 

cb  =  a  +  T^,(t-273.15)  +  'F,C,.  (2.2) 

The  temperature  and  salinity  corrected  absorption  coefficient  <X>  is  introduced  in 
Equation  (2.2).  The  absorption  coefficient  a  for  a  wavelength  of  620  nm  in  salt  water  at 
20  °C  is  0.002755  m'^  [32].  The  temperature  dependence  Wt  is  0.000539  m'^  °C'\  and  the 
salinity  dependence  Ws  is  0.0000838  m'^g'^E  for  a  wavelength  of  620  nm  as  provided  by 
Rottgers  et  al.  [41].  Substituting  these  values  into  Equation  (2.2),  we  get  the  corrected 
absorption  coefficient  at  620  nm  as 

=  0.002755  +  0.000539(t  -  273. 15)  +  0.0000838  x  C,  .  (2.3) 
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The  water  temperature  t  is  give  in  Kelvin  and  Cs  is  the  salt  concentration  of  the 


water  in  g/L. 

The  properties  of  light  propagation  in  the  ocean  water  are  the  basis  of  the 
development  of  the  techniques  for  estimating  depth  in  three  dimensions  (3D)  underwater 
scenes. 
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III.  DEVELOPMENT  OE  PROPOSED  TECHNIQUE 


The  natural  properties  of  light  described  in  Chapter  II  may  be  used  to  compare 
two  underwater  images  to  provide  a  three-dimensional  representation  of  the  environment 
surrounding  the  AUV.  This  technique  not  only  does  not  require  significant  processing 
power  but  is  also  passive  and  does  not  require  energy  sources,  such  as  sound  or  artificial 
light.  A  camera,  filter,  and  natural  light  are  all  that  are  required. 

Accurate  results  do  require  the  temperature  and  salinity  to  be  known,  and  they  are 
highly  dependent  on  turbidity,  or  the  concentration  of  particulate  in  the  water.  This 
method  is  considered  a  type  of  visual  odometry  because  it  analyzes  camera  images  to 
process  an  environment.  The  temperature  and  salinity  corrected  absorption  coefficient  0 
has  been  substituted  into  Equation  (2.1)  and  rearranged  to  solve  for  a  distance  d\ 


(lA 

d  =  -\ 

in 

[o) 

UJ 

If  precise  wavelength- specific  luminous  intensities  can  be  obtained  just  below  the 
surface  of  the  ocean  Ioa  and  at  the  subject  1a  then  the  depth  of  the  object  d-A  may  be 
determined  based  on  Equation  (3.1)  as  shown  in  Eigure  3.  A  form  of  Equation  (3.1)  is 
often  used  in  determining  where  light-dependent  organisms  may  be  found  based  on  their 
food  requirements  [42]-[44].  Similarly,  if  the  wavelength- specific  luminous  intensities 
can  be  obtained  at  the  subject  Iob  and  at  the  camera  h,  then  the  distance  from  the  subject 
to  the  camera  ds  can  also  be  determined  from  equation  (3.1),  also  shown  in  Eigure  3.  The 
total  distance  that  the  light  has  traveled  through  the  water  includes  the  depth  of  the  object 
and  the  range  to  the  object:  distance  =  depth  -l-  range .  This  corresponds  to  adding  Ja  and 
dfi  from  Eigure  3  together  to  get  the  total  distance  the  light  has  traveled  through  the  water. 
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Figure  3.  Image  Showing  Distances  from  Surface  to  Subject  and  to  Camera 

If  the  light  intensity  decays  at  the  same  rate  with  distance,  then  the  color  will 
appear  the  same  but  with  decreased  brightness  (lower  luminous  intensity).  As  described 
by  the  Beer-Lambert  law  in  Chapter  II,  the  light  intensity  at  different  wavelengths 
experience  different  exponential  decays  and  is  perceived  as  a  shift  in  color  from  reds  to 
blues  and  greens. 

An  image  is  stored  as  a  three-dimensional  matrix  of  RGB  values,  which  together 
describe  a  color  for  each  pixel.  As  light  travels  through  the  water,  these  RGB  values 
decay  toward  the  green-blue  side  of  the  color  triangle.  An  observer,  or  in  this  case  a 
camera,  sees  a  different  color  because  the  wavelength-specific  luminous  intensities  decay 
unevenly  causing  the  color  shift.  As  the  red  is  absorbed  first  there  is  a  shift  in  color 
towards  blue  and  green.  Studying  the  shift  in  color  provides  some  useful  information  that 
is  comparable  to  using  the  luminous  intensities. 

Filters  are  widely  used  in  underwater  photography.  They  are  designed  to  correct 
for  the  shift  in  color.  Filters  work  by  blocking  some  wavelengths  (mostly  blues  and 
greens)  from  reaching  the  camera  sensor.  They  do  not  entirely  block  these  wavelengths 
but  allow  more  red  wavelengths  to  reach  the  camera  to  help  restore  the  balance  of  color. 
Blocking  some  of  the  blue  and  green  wavelengths  causes  a  shift  in  color  back  toward  a 


14 


natural  balance  of  colors.  Restoring  the  balance  of  color  comes  at  a  cost.  By  blocking 
some  of  the  light,  the  image  appears  darker,  but  the  colors  appear  more  natural. 

Filters  are  also  limited  in  use  by  the  decay  of  the  red  wavelengths.  At  deeper 
depths  or  when  photographing  subjects  that  are  farther  away,  the  red  wavelengths  decay 
entirely  and  the  filter  is  not  useful.  An  image  captured  with  a  filter  will  appear  as  close  to 
the  correct  color,  although  at  a  lower  intensity,  because  the  RGB  values  for  each  pixel 
have  been  restored  to  a  proper  balance.  The  filtered  image  may  be  used  as  the  initial  color 
or  the  properly  balanced  color,  while  the  unfiltered  image  shows  how  much  the  color  has 
shifted  over  an  unknown  distance.  The  Beer-Lambert  law  uses  light  intensities  to 
determine  a  range.  Having  the  proper  color  representation  does  not  give  the  proper 
luminous  intensity.  Determining  how  much  the  color  has  shifted  only  provides  a  relative 
distance  between  objects  in  the  image  when  used  in  Equation  (3.1)  rather  than  an 
absolute  distance  from  the  camera  to  the  objects. 

Most  visual  and  laser  based  methods  of  underwater  ranging  prefer  a  wavelength 
that  has  the  least  attenuation.  Choosing  light  in  the  blue-green  region  ensures  that  the 
light  will  travel  farther,  thus  illuminating  objects  farther  away  or  providing  a  higher 
chance  of  reaching  the  collector.  For  this  technique,  light  in  the  red  region  is  used 
because  it  has  the  highest  attenuation  over  a  short  distance.  This  is  done  to  create  the 
highest  contrast  between  the  initial  intensity  and  final  intensity  over  the  smallest  distance 
or,  in  this  case,  the  greatest  color  shift  over  the  smallest  distance. 

The  blue  and  green  wavelengths  experience  a  much  slower  exponential  decay. 
They  are  assumed  to  remain  constant  over  the  short  distances  analyzed.  An  RGB  value  of 
(1,0,0),  corresponding  to  a  wavelength  of  620  nm  (bright  red),  is  selected  as  a  reference 
for  convenience  because  it  is  on  one  comer  of  the  RGB  color  space.  This  wavelength  is 
known  to  rapidly  decay  towards  the  blue-green  side  of  the  triangle  in  Figure  2,  so  it 
provides  some  color  shift.  For  example,  a  pixel  with  a  value  of  (1,0,0)  indicates  that  no 
red  has  been  lost;  therefore,  the  object  has  to  reflect  red  (the  object  has  red  (1,0,0)  or 
white  (1,1,1)  in  its  color)  and  be  directly  in  front  of  the  camera  and  just  beneath  the 
surface  (no  loss  due  to  absorption).  If  that  object  is  moved  down  the  water  column  or 
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away  from  the  camera,  then  it  experiences  a  color  shift.  Measuring  that  color  shift  reveals 
how  far  the  light  has  traveled  underwater. 

The  difference  in  red  pixel  values  between  two  images,  one  filtered  and  the  other 
unfiltered,  taken  at  the  same  location  may  be  used  to  determine  the  relative  distance 
between  objects  within  the  frame.  Substituting  filtered  and  unfiltered  red  pixel  values  for 
initial  and  final  intensities  into  Equation  (3.1),  we  get 


where  represents  the  R  matrix  in  the  unfiltered  image,  Rf  is  the  R  matrix  in  the  filtered 
image,  and  d  has  been  replaced  with  dr  to  denote  a  relative  distance  between  objects 
within  the  image. 

A  boundary  condition  occurs  when  the  filtered  pixel  value  contains  no  red  (0,0,0) 
because  taking  the  natural  log  of  0  yields  -oo.  This  indicates  one  of  two  situations,  either 
there  was  no  red  reflected  from  the  object  (the  initial  intensity  for  red  wavelengths  was  0) 
or  all  of  the  red  wavelengths  have  fully  decayed  (the  light  has  traveled  far  enough  to  fully 
decay). 

In  the  next  chapter  we  demonstrate  the  effectiveness  of  this  technique  by  applying 
it  to  a  number  of  different  depth  and  range  conditions. 
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IV.  EXPERIMENTAL  DESIGN  AND  DATA  COLLECTION 


This  new  technique  is  dependent  on  colors  and  specifically  comparing  colors 
between  images.  Using  different  cameras  to  take  some  of  the  images  or  using  different 
computers  to  process  the  results  may  introduce  unnecessary  error  because  of  different 
RGB  color  spaces;  therefore,  all  images  were  collected  using  the  same  camera  setup  in 
the  same  general  location  by  the  same  operator  over  consecutive  days  to  reduce 
variations.  All  processing  was  done  on  the  same  computer  using  the  same  version  of 
software. 

A.  EQUIPMENT  SELECTION  AND  SETUP 

Initially,  data  collection  was  attempted  in  a  tank  and  a  pool.  Data  collection  in  a 
tank  offered  the  challenge  of  controlling  the  light  source.  The  tank  had  glass  on  all  sides 
which  allowed  light  in  from  all  angles,  preventing  successful  data  collection.  The  glass 
also  has  some  absorbance  which  would  have  affected  the  results;  therefore,  a  tank  is  not 
desirable  for  data  collection.  A  pool  was  also  attempted.  Light  only  penetrated  the  surface 
of  the  water;  however,  most  pools,  including  the  one  used,  are  painted  white,  which 
allows  significant  reflection  from  the  bottom  and  sides.  The  reflectivity  of  light 
underwater  from  surfaces  such  as  a  sandy  bottom  has  been  studied  previously  by  Lee  et 
al.  [45],  Mobley  et  al.  [46]  and  English  and  Carder  [47].  The  reflectivity  of  the  light  off 
the  bottom  and  sides  of  a  white  pool  significantly  impacts  the  results  and  is  also  not 
desirable. 

Further  investigation  of  a  suitable  location  to  gather  data  suggested  a  gently 
sloping  ocean  floor  at  depths  of  5-12  m  with  visibility  that  exceeded  20  m  to  ensure 
absorption  of  the  desired  wavelength  dominated  rather  than  scattering.  A  sandy  bottom 
would  also  result  in  reflection  like  in  the  pool,  so  a  rocky  bottom  is  preferred.  A  high 
tidal  current  and  large  sea  state  will  cause  difficulty  in  obtaining  proper  images,  so  an 
area  with  low  current  is  preferable.  After  careful  consideration,  a  location  on  the  West 
side  of  Oahu  in  the  Hawaiian  Islands  was  selected. 
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A  GoPro  Hero  4  was  selected  as  a  camera.  The  Hero  4  is  easy  to  use,  records  high 
quality  High  Definition  (HD)  video,  and  is  easily  accessible.  It  was  mounted  on  a  tripod 
and  taken  to  depths  of  5-10  m  of  water  and  placed  at  various  distances  from  natural  and 
manmade  objects.  Video  footage  was  taken  at  each  location  with  two  filters  (Flip4 
“Dive”  and  “Deep”)  and  without  a  filter.  Data  was  collected  during  five  dives  over  a  two- 
day  period  at  various  times  of  day  and  night.  Diver  one  set  up  the  tripod  and  operated  the 
filters  during  all  data  collection.  Diver  two  measured  the  distance  to  a  known  object  for 
reference  for  each  dataset.  Video  footage  was  processed  using  Pinnacle  19  to  obtain  still 
images.  Images  were  cropped  to  ensure  each  pair  of  filtered  and  unfiltered  images  had  the 
same  field  of  view.  A  filtered  image  and  the  corresponding  unfiltered  image  were 
processed  as  a  pair  using  MATLAB  R2015b  Image  Processing  toolbox  and  user 
generated  code  found  in  the  Appendix. 

Water  temperature,  salinity,  and  density  were  measured  using  an  RBR  Concerto 
as  shown  in  Figure  4.  Averages  for  each  dive  were  calculated  and  compiled  in  Table  2.  A 
temperature  of  27.55  °C  was  used  as  the  average  temperature,  34.60  PSU,  or  g/kg,  was 
used  as  the  average  salinity,  and  22.27  kg/m  was  used  as  the  average  density. 


Figure  4.  RBR  Concerto  Used  to  Measure  Temperature  and  Salinity. 

Source:  [48]. 
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Table  2.  Averaged  Data  from  RBR  Probe 


Temp 

Salinity 

Density 

Cs 

Dive  # 

°C 

PSU 

kg/m^ 

g/L 

1 

27.41805 

33.84613 

21.73631 

0.73569 

2 

27.72177 

34.70507 

22.28743 

0.773487 

3 

27.5493 

34.7634 

22.38712 

0.778252 

4 

27.42537 

34.75553 

22.42119 

0.77926 

Average 

27.54915 

34.60401 

22.26674 

0.770519 

Using  these  average  values  in  Equation  (2.3)  produces  a  temperature  and  salinity 
corrected  absorption  coefficient  of  0.00689  m'\  Substituting  this  into  Equation  (3.2) 
produces 


J= -0.145 17  In 


(4.1) 


with  the  relative  distance  dr  given  in  mm.  Image  pairs  were  captured  as  described  above 
producing  a  pair  of  RGB  matrices.  The  emphasis  of  this  experiment  is  to  get  range 
information  for  short  distances  (less  than  10  m)  so  the  analysis  focuses  on  the  R  matrices. 
Eor  each  (x,y)  pixel,  the  R  value  from  the  unfiltered  image  was  divided  by  the  R  value 
from  the  filtered  image.  This  produced  a  new  matrix  of  relative  distances  for  each  (x,y) 
pixel  value.  This  matrix  is  represented  as  a  three-dimensional  wire-mesh  with  colored 
peaks  indicating  distances. 

A  dark  blue  indicates  a  low  value  or  no  object  detected.  A  bright  yellow  indicates 
a  high  value  or  close  object.  These  are  still  relative  values,  so  a  bright  yellow  object  may 
not  be  close  though  it  may  be  closer  than  all  other  detected  objects.  The  boundary 
condition  mentioned  above  creates  unnecessary  peaks  which  may  skew  the  results.  As  a 
result,  code  was  added  to  search  for  pixels  whose  value  exceed  a  threshold,  and  those 
pixels  were  set  to  a  nominal  value  to  avoid  skewing  the  results. 


B.  DATA  PROCESSING 

The  theoretical  maximum  distance  Dmax,t  traveled  by  red  wavelengths  of  light  in 
clear  water  is  10  m.  The  first  pair  of  images  (Eigure  5  and  Eigure  6)  were  taken  at  a  depth 
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of  8.3  m  and  approximately  2.0  m  from  the  subject  to  the  camera,  so  the  light  has  traveled 
a  total  of  10.3  meters  through  the  water.  This  image  pair  is,  therefore,  near  the  theoretical 
maximum  distance  for  red  light. 

An  unfiltered  image  taken  approximately  2.0  m  from  the  subject  at  a  depth  of  8.3 
m  is  shown  in  Figure  5.  This  image  represents  a  matrix  of  light  that  has  lost  intensity  and 
experienced  a  color  shift  as  the  light  has  traveled  from  the  surface  to  the  subject  and  from 
the  subject  to  the  camera.  This  matrix  does  not  accurately  depict  the  change  in  luminous 
intensity  but  rather  the  shift  in  color  as  the  wavelengths  of  light  have  been  absorbed.  This 
image  appears  to  be  washed  out,  mostly  showing  greens  and  blues  and  is  common  among 
amateur  underwater  photographers. 


Figure  5.  Unfiltered  Image  8.3  m  Deep,  Range  of  2.0  m  to  Subject 

A  filtered  image  is  shown  in  Figure  6  that  was  taken  from  the  same  location  as  the 
image  in  Figure  5.  This  image  represents  a  close  approximation  of  the  original  colors  of 
the  subject.  This  does  not  accurately  represent  the  initial  luminous  intensities  from  the 
Beer- Lambert  Law  but  instead  represents  the  color  balance  restored  by  using  the  filter. 
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Figure  6.  Filtered  Image  8.3  m  Deep,  Range  of  2.0  m  to  Subjeet 


By  subtraeting  the  unfiltered  image  from  the  filtered  image,  we  see  how  the  filter 
altered  the  image.  A  pixel-by-pixel  subtraetion  of  Figure  5  from  Figure  6  produees  the 
results  shown  in  Figure  7. 


Figure  7.  Image  Showing  Differenee  between  Figure  5  and  Figure  6 
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After  fully  processing  the  images,  the  results  are  displayed  as  a  3D  wire-mesh  in 
Figure  8  and  Figure  9.  For  each  (x,y)  pixel  in  the  image,  a  value  is  given  to  describe  the 
relative  distance.  A  brighter  yellow  (taller  peak)  shows  objects  that  are  closer,  while 
darker  blues  (valleys)  show  objects  that  are  farther  away.  The  coral  is  visible  with  yellow 
peaks  indicating  where  the  coral  sticks  out  farther  or  is  closer  to  the  camera.  The  lines  of 
raised  iron  are  also  visible  on  the  right-hand  side  of  the  image  indicating  where  the  wall 
protrudes.  This  can  also  be  converted  to  a  3D  point  cloud  for  use  in  navigating 
underwater  vehicles. 


1500  1000  500  0 

X  (pixel) 

Figure  8.  3D  Representation  of  Figure  5  and  Figure  6 
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1500 

Figure  9.  3D  Representation  of  Figure  5  and  Figure  6  Rotated  to  Show  Depth 


Zooming  in  on  specific  features,  such  as  the  coral  in  the  middle  of  the  image  near 
the  top  of  the  wall,  demonstrates  the  resolution  of  this  technique.  The  post-processed 
image  and  the  filtered  image  are  shown  in  Figure  10  for  comparison.  It  is  easy  for  the 
human  eye  to  imagine  the  coral  protruding  in  a  photograph  but  difficult  to  quantify  how 
far  it  protrudes.  It  is  even  more  difficult  to  convey  this  obstacle  to  an  underwater  vehicle. 
This  technique  can  quantify  these  objects  and  be  used  in  navigating  an  underwater 
vehicle. 

The  same  subject  photographed  at  a  range  of  5.0  m  and  a  depth  of  8.3  m  is  shown 
in  Figure  11  and  Figure  12.  Adding  these  distances  together  gives  a  total  distance  of  13.3, 
which  is  33%  larger  than  the  theoretical  maximum  distance  Dmax.t  that  the  red 
wavelengths  can  travel.  The  expected  result  is  less  resolution  because  some  of  the  filtered 
pixel  values  have  reached  zero.  The  unfiltered  image  is  shown  in  Figure  11,  and  the 
filtered  image  is  shown  in  Figure  12. 
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Figure  10.  Comparison  of  Processed  3D  Image  of  Coral  to  Original  Coral  Image 

with  Features  Highlighted 


Figure  11.  Unfiltered  Image  8.3  m  Deep,  Range  of  5.0  m  to  Subject 
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Figure  12.  Filtered  image  8.3  m  Deep,  Range  of  5.0  m  to  Subject 


As  expected,  the  same  wall  photographed  at  5.0  m  shows  significantly  less 
resolution  than  the  images  at  2.0  m  even  when  cropped  to  cover  the  same  area.  All  the 
intensities  have  decreased  as  the  light  has  traveled  farther  through  the  water.  With  a 
lower  overall  intensity,  it  is  more  difficult  to  distinguish  details  within  the  remaining 
values.  This  can  be  seen  in  Figure  13.  The  image  appears  darker  overall. 
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z(mm)  y  (pixel) 


There  are  many  pixels  that  reflected  red  light,  but  much  of  that  light  does  not 
make  it  to  the  camera  such  that  even  with  the  filter  they  appear  as  (0,0,0),  or  far  away. 
This  can  be  seen  in  the  final  processed  image  in  Figure  14  and  Figure  15. 
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Figure  14.  3D  Representation  of  Figure  11  and  Figure  12 
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Figure  15.  3D  Representation  of  Figure  11  and  Figure  12  Rotated  to  Show  Depth 
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When  comparing  the  features  in  Figure  10,  this  same  coral  has  significantly  less 
resolution  as  seen  in  Figure  16.  The  same  features  are  still  visible  but  less  prominent. 
Some  red  did  still  reach  the  camera,  so  13.3  m  is  not  an  absolute  maximum  for  the  red 
wavelengths  to  travel.  It  does  indicate  that  using  red  wavelengths  is  less  accurate  at  this 
distance. 
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Figure  16.  Comparison  of  Processed  3D  Image  of  Coral  with 

Original  Coral  Image 


There  exists  a  local  maximum  distance  D,„ax,i  which  depends  on  the  temperature 
and  salinity  as  well  as  the  particulate  concentration.  This  local  maximum  may  be 
significantly  smaller  than  the  theoretical  maximum,  especially  because  of  higher 
particulate  concentration.  The  local  maximum  where  this  technique  remains  effective  lies 
somewhere  between  10.3  m  and  13.3  m.  This  local  limit  can  be  quantified  with  further 
experimentation  but  will  vary  by  location  and  conditions.  In  situations  where  high 
resolution  may  be  sacrificed  for  speed,  this  level  of  accuracy  may  be  sufficient.  To 
navigate  an  underwater  vehicle,  it  may  be  sufficient  to  know  that  there  is  an  object 
present  without  knowing  the  detailed  features  of  the  object. 
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Images  taken  at  deeper  depths  must  be  eloser  to  the  objeets  to  remain  inside  the 
loeal  maximum  Dmax.i-  An  objeet  photographed  at  2.0  m  and  a  depth  of  10.0  m  is  shown 
in  Figure  17  and  Figure  18.  This  means  that  the  total  distanee  the  light  has  traveled 
underwater  is  12.0  m  which  is  still  20%  larger  than  the  theoretical  maximum  distance 
Dmax,t-  This  image  pair  shows  less  contrast  like  the  images  shown  in  Figures  11-15.  An 
unfiltered  image  taken  2.0  m  away  at  a  depth  of  10.0  m  is  shown  in  Figure  17,  and  the 
corresponding  filtered  image  is  shown  in  Figure  18. 


Figure  17.  Unfiltered  Image  10.0  m  Deep,  Range  of  2.0  m  to  Subject 
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Figure  18.  Filtered  Image  10.0  m  Deep,  Range  of  2.0  m  to  Subjeet 

Subtraeting  the  image  in  Figure  17  from  the  image  in  Figure  18  produees 
Figure  19,  which  shows  the  effect  of  the  filter.  This  image  is  darker,  like  Figure  13. 


Figure  19.  Image  Showing  Difference  Between  Images  in 

Figure  17  and  Figure  18 
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The  processed  image  from  a  depth  of  10.0  m  and  a  range  of  2.0  m  is  shown  in 
Figure  20  and  Figure  21.  The  outline  of  the  subject  is  clearly  visible,  but  there  is  not  as 
much  depth  to  the  3D  image  as  there  was  in  Figure  8  and  Figure  9.  The  fish  standout  as 
the  closest  (and  perhaps  the  brightest  colored)  objects  in  the  frame. 
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Figure  20.  3D  Representation  of  Figure  17  and  Figure  18 


Figure  21.  3D  Representation  of  Figure  17  and  Figure  18  Rotated  to  Show  Depth 
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Images  taken  at  a  shallower  depth  (5.0  m)  and  range  of  2.0  m  are  eompared  in 
Figures  22-26.  The  unfiltered  image  at  this  depth  and  range  eombination  is  shown  in 
Figure  22  with  the  eorresponding  filtered  image  shown  in  Figure  23. 


Figure  22.  Unfiltered  Image  5.0  m  Deep,  Range  of  2.0  m  to  Subjeet 


Figure  23.  Filtered  Image  5.0  m  Deep,  Range  of  2.0  m  to  Subject 
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At  this  depth  and  range,  the  total  distanee  should  be  well  within  the  theoretieal 
maximum  distanee  Dmax,t-  Indeed,  proeessing  this  image  pair  yields  good  eontrast 
between  objeets  in  the  frame  as  seen  in  Figures  24  and  25.  This  eontrast  appears  to  be 
better  than  that  in  Figure  8  beeause  the  red  light  has  not  traveled  as  far.  There  is, 
however,  some  saturation  evident  in  the  lower  left  quadrant  of  Figure  24.  This  area  has 
been  blown  up  to  show  the  detail  in  Figure  26.  The  reflectance  from  the  sand  is  visible, 
causing  a  “false  return.”  This  was  anticipated  when  analyzing  images  with  significant 
amounts  of  sand  visible  as  predicted  by  English  and  Carder  [47]. 
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Figure  24.  3D  Representation  of  Figure  22  and  Figure  23 
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Figure  25.  3D  Representation  of  Figure  22  and  Figure  23  Rotated  to  Show  Depth 


Figure  26.  Exploded  View  of  Lower  Left  Corner  of  Ligure  24  Compared  to  the 

Original  Liltered  Image  in  Ligure  23 
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Images  taken  at  a  depth  of  5.0  meters  and  range  of  8.3  meters  from  the  subject  are 
shown  in  Figure  27  and  Figure  28,  respectively.  This  distance  is  known  to  be  outside  the 
local  maximum  Dmax.i  but  presents  some  interesting  information.  The  unfiltered  image  is 
shown  in  Figure  27,  and  the  corresponding  filtered  image  is  shown  in  Figure  28. 


Figure  28.  Filtered  Image  5.0  m  Deep,  Range  of  8.3  m  to  Subject 
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The  processed  image  from  the  pair  shown  in  Figures  27  and  28  is  shown  in 
Figure  29.  The  bottom  reflectance  from  the  sand  is  again  visible  across  the  bottom  in 
Figures  29  and  30.  Additionally,  the  line  used  to  measure  distance  to  the  objects  did  not 
clear  the  picture  and  shows  up  as  a  bright  yellow  line  on  the  right-hand  side  of  the  image. 
It  is,  in  fact,  the  closest  object  in  the  image  and  is  also  red  in  color.  The  limitation  of 
using  a  red  wavelength  is  also  visible  as  the  coral  in  the  middle  of  the  raw  images  does 
not  appear  in  the  processed  image.  There  is  a  distinct  gradient  seen  along  the  right-hand 
side  of  the  image.  Objects  in  the  lower  right-hand  corner  are  within  the  local  maximum 
Dmax,i-  and  appear  to  have  decent  contrast.  Objects  that  are  farther  away  start  to  lose 
contrast  and  eventually  fade  from  view. 
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Figure  29.  3D  Representation  of  Figure  27  and  Figure  28 
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Figure  30.  3D  Representation  of  Figure  27  and  Figure  28  Rotated  to  Show  Depth 

Another  observation  from  this  image  is  that  with  significant  visible  water 
overhead  it  is  possible  to  see  striations  in  the  water  due  to  the  flat  port  used  to  capture  the 
images.  This  may  be  due  to  the  polarized  nature  of  light  in  water  as  previously  studied  by 
Cronin  and  Shashar  [49]  and  many  others. 

C.  GREEN  AND  BLUE  MATRIX  ANALYSIS 

Similar  analysis  was  performed  on  the  green  and  blue  matrices  from  the  images  in 
Figures  5  and  6.  The  temperature  and  salinity  corrected  absorption  coefficients  for  the 
other  two  comers  of  the  Adobe  RGB  triangle,  green  (510  nm)  and  blue  (440  nm),  were 
calculated  using  Equation  (2.3)  and  data  provided  by  Rottgers  et  al.  [41].  Results  are 
shown  in  Table  3  for  all  three  wavelengths. 
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Table  3.  Temperature  and  Salinity  Corrected  Absorption  Coefficients. 

Adapted  from  [32],  [41]. 


Wavelength 

440  nm 

510  nm 

620  nm 

a  (1/m) 

6.35x10'^ 

3.25x10"® 

2.76x10® 

i]js(l/m*L/g) 

2.22x10-® 

1.75x10® 

8.38x10® 

i]jT(l/m*l/K) 

2.96x10"® 

7.71x10"® 

5.39x10'* 

c|)  =  a  +  i]jT(t- 273.15) +  i]js(Cs) 

c|)(l/m) 

6.39x10"® 

3.31x10"^ 

6.89x10® 

The  values  from  Table  2  may  be  substituted  into  Equation  (3.2)  for  O510  and  ^440 
to  obtain 


and 


d= -30.23  in 


v^/y 


d  =-156.511n 


_ u 

\^f  J 


(4.2) 


(4.3) 


where  represents  the  green  pixel  values  from  the  unfiltered  image,  G/  represents  the 
green  pixel  values  from  the  filtered  image,  Bu  represents  the  blue  pixel  values  from  the 
unfiltered  image,  and  5/ represents  the  blue  pixel  values  from  the  filtered  image. 

The  green  and  blue  matrices  were  analyzed  using  Equations  (4.2)  and  (4.3) 
similar  to  the  way  the  red  matrix  was  analyzed  with  Equation  (4.1).  As  expected,  the 
matrix  of  green  values  shows  very  little  change  over  a  short  distance,  as  seen  in 
Eigure  31. 
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Figure  31.  3D  Representation  Based  on  the  Green  Pixel  Values  from 

Figures  5  and  6 
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There  is  a  surprisingly  large  color  shift  in  the  blue  matrix  values  for  this  image 
pair  as  seen  in  Figure  32.  In  this  image  it  is  easier  to  distinguish  the  open  ocean  above  the 
coral  from  the  edge  of  the  coral.  The  results  from  the  blue  matrix  may  be  better  than  the 
results  from  the  red  matrix  in  determining  a  safe  place  for  navigating  an  underwater  robot 
in  this  case.  The  higher  wavelengths  seem  to  experience  significant  absorption  over  short 
distances,  especially  the  purples  and  deep  blues  as  seen  in  the  left-hand  side  of  Figure  16. 
This  shows  that  both  the  red  and  blue  comers  of  the  Adobe  RGB  triangle  shift  toward  the 
green  corner  in  Figure  17. 
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Figure  32.  3D  Representation  Based  on  the  Blue  Pixel  Values  from 

Figures  5  and  6 
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Comparing  the  results  from  analyzing  the  green  and  blue  matriees  for  the  images 
in  Figure  27  and  28  taken  at  a  depth  of  5.0  m  and  range  of  8.3  m  produees  the  3D 
representations  shown  in  Figure  33  and  Figure  34,  respectively.  The  red  line  used  to 
measure  distances  is  no  longer  dominant  and  is  now  barely  visible.  The  bottom  reflection 
from  the  sand  is  also  notably  absent.  The  green  matrix  provides  little  value,  but  the  blue 
matrix  shows  the  coral  on  the  left-hand  side  of  the  image  as  well  as  the  coral  in  the 
middle  of  the  image  that  did  not  show  up  when  analyzing  the  red  matrix. 


Figure  33.  3D  Representation  Based  on  the  Green  Pixel  Values  from  Figure  27 

and  Figure  28 
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Figure  34.  3D  Representation  Based  on  the  Blue  Pixel  Values  from  Figures  27 

and  28 


As  previously  mentioned,  the  RGB  color  space  adds  red,  green,  and  blue  values  to 
produce  a  color.  Splitting  a  color  into  its  red,  green,  and  blue  components  allows  analysis 
of  how  each  one  changes  as  it  passes  through  water.  The  green  component  changes  very 
little,  as  expected,  because  its  exponential  decay  is  significantly  slower  than  the  other 
two.  The  blue  and  red  components  both  provide  useful  information  in  analyzing  a  scene. 
Adding  the  3D  results  from  analyzing  the  red,  green,  and  blue  matrices  back  together  to 
reform  an  RGB  representation  produces  little  added  value  and  is  not  recommended,  as 
seen  in  Figure  35. 
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This  image  shows  a  comparison  of  green  (a),  blue  (b),  red  (c),  and  a  combination 
of  red  +  green  +  blue  (d)  3D  representations  of  Figure  10.  It  is  clear  from  the  blue  matrix 
in  Figure  35(b)  where  the  clear  blue  water  is  compared  to  the  coral.  The  outline  of 
individual  coral  is  more  visible  from  the  red  matrix  in  Figure  35(c).  Combining  the 
images  make  everything  less  visible  as  seen  in  Figure  35(d). 


a)  b) 


Figure  35.  3D  Representations  of  Figure  10  Using  Green  (a),  Blue  (b),  Red  (c), 

and  Red  +  Green  +  Blue  (d)  Values 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  CONCLUSIONS 

A  new  technique  has  been  developed  for  improving  underwater  robot  vision.  This 
technique  is  vision-based  and  only  requires  a  camera,  a  filter,  and  sufficient  natural  light. 
By  comparing  two  raw  images,  one  filtered  and  one  unfiltered,  we  can  generate  a  detailed 
three-dimensional  image  showing  the  relative  distance  between  objects  in  a  scene.  This  is 
done  by  separating  the  color  in  each  pixel  into  its  respective  red,  green,  and  blue  values. 
The  red  and  blue  values  shift  toward  the  green  comer  in  the  RGB  color  space  as  the  light 
travels  through  water.  Analyzing  the  amount  these  values  shift  approximates  relative 
distances  between  objects  in  the  frame.  It  is  possible  to  capture  and  process  these  images 
to  determine  these  relative  ranges  using  commercially  available  equipment. 

Absolute  ranges  from  a  vehicle  to  objects  in  a  scene  are  more  important  and  were 
the  original  intent  of  this  work.  This  technique  alone  cannot  determine  absolute  ranges 
but  does  provide  useful  information  and  can  still  be  used  to  provide  robot  vision.  This 
technique  can  also  be  combined  with  other  sensors  or  techniques  and  further  develop  into 
absolute  ranges.  For  example,  it  can  be  combined  with  a  single-point  laser  rangefinder. 
Knowing  the  absolute  range  to  a  single  point  reveals  the  range  to  all  other  points  based  on 
the  relative  ranges  provided  by  this  technique.  Another  possibility  is  to  employ  this 
technique  in  stereo  or  in  combination  with  motion.  Comparing  results  between  two 
cameras  with  known  spacing  may  provide  additional  range  information.  Similarly, 
processing  sequential  images  and  analyzing  the  optical  flow  field  may  also  provide 
additional  range  information.  This  is  known  as  egomotion  [18]. 

A  red  wavelength  was  chosen  based  on  the  known  properties  of  short- 
wavelengths  in  water.  This  wavelength  does  indeed  provide  useful  relative  range  data  for 
a  given  scene.  Analyzing  the  blue  wavelength  also  provides  useful  data  and  should  also 
be  used  to  establish  robot  vision.  The  green  wavelength  is  not  useful  over  short  distances 
but  may  prove  useful  over  longer  distances. 
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The  upward  reflectance  is  a  significant  issue  for  areas  with  a  sandy  bottom. 
Analyzing  the  change  in  blue  values  may  be  able  to  counter  this  negative  effect.  One  way 
this  may  be  useful  is  to  analyze  the  image  in  sections.  There  are  many  edge  detection 
algorithms  that  are  suitable  for  processing  images.  These  algorithms  may  be  applied  to 
the  results  from  the  blue  analysis  to  determine  which  portions  of  the  image  should  be 
analyzed  using  the  red  values  and  which  portions  should  be  ignored  (open  ocean,  sand, 
etc.).  For  robot  vision,  knowing  where  the  open  ocean  is  compared  to  objects  may  be 
sufficient  to  establish  a  low  level  of  autonomy. 

In  conclusion,  this  is  an  exciting  new  method  in  underwater  robot  vision  that  has 
not  been  previously  attempted.  Comparing  an  unfiltered  image  with  a  filtered  image 
produces  relative  range  information.  Absolute  ranges  may  be  determined  when 
combining  this  technique  with  other  sensors  or  with  future  development.  Red  and  blue 
frequencies  are  used  for  better  ranging  at  short  distances. 

B.  RECOMMENDATIONS 

Additional  experiments  are  recommended  under  similar  conditions  but  with  an 
underwater  laser  rangefinder.  This  will  provide  an  absolute  range  to  a  single  point  and 
conceivably  every  point  in  the  image.  An  underwater  laser  line  scanner  can  also  be  used 
to  verify  calculations  and  prove  this  technique.  Mounting  multiple  cameras  and  analyzing 
a  sequence  of  images  using  egomotion  is  also  recommended  to  provide  additional  depth 
perception  to  image  results 

Performing  additional  experiments  are  also  recommended,  including  the  use  of  a 
variety  of  filters  at  increased  depths  and  ranges  to  test  the  limit  of  usability  for  this 
technique.  Comparing  results  between  similar  conditions  with  different  filters  may 
provide  additional  range  information  or  verify  current  calculations.  Recent  work  has  been 
done  to  develop  adaptive  lighting  techniques  that  render  objects  as  they  would  appear  in 
air  [29].  Comparing  unfiltered  images  with  images  taken  with  adaptive  lighting  would 
seem  to  provide  better  results.  The  filtered  image  approximates  true  colors  for  objects 
underwater  but  is  imprecise  because  each  filter  can  only  account  for  a  small  range  of 
wavelengths. 
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Once  absolute  ranges  can  be  obtained,  a  small  form-factor  apparatus  should  be 
designed  to  utilize  this  technique  in  real  time.  This  apparatus  should  capture  both  filtered 
and  unfiltered  images,  process  the  image  pairs,  and  store  or  convey  the  results  such  as 
building  a  3D  point  cloud.  The  next  step  would  be  to  design  a  controller  to  drive  the 
vehicle  toward  a  goal.  This  would  require  a  path  planning  algorithm  using  the 
information  form  the  point  cloud. 
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APPENDIX.  MATLAB  CODE 


%This  program  takes  two  images  captured  from  the  same  location,  one  taken 

%with  a  red  filter  and  one  without  a  red  filter,  and  compares  the  shift  in 

%color.  The  images  are  processed  using  a  form  of  Beer's  Law  as  discussed  in 

%the  paper.  Results  are  displayed  in  Figures  as  described  below. 

clear  all;close  all;clc; 

unf_img  =  'Dive3_Shallow_Dl_NF.jpg'; 

unfiltered_frame  =  double(imread(unf_img));  %unfiltered  image 
%Image  taken  without  a  filter. 
unfiltered_size  =  size(unfiltered_frame); 

unfiltered_frame  =  imcrop(unfiltered_frame,[0,0,unfiltered_size(2),unfiltered_size(l)]); 
unfiltered_pixels  =  reshape(unfiltered_frame,  unfiltered_size(l)*unfiltered_size(2), 
unfiltered_size(3)); 

unfiltered  =  reshape(unfiltered_pixels,  unfiltered_size); 

fil_img  =  'Dive3_Shallow_Dl_F2.jpg'; 

filtered_frame  =  double(imread(fil_img));  %filtered  image 

%Image  taken  with  a  filter. 

filtered_size  =  size(filtered_frame); 

if  filtered_size(2)  ~=  unfiltered_size(2) 

filtered_frame  =  imcrop(filtered_frame,[0,0,unfiltered_size(2),unfiltered_size(l)]); 
filtered_size  =  unfiltered_size; 
end 

%The  images  must  be  the  same  size  for  comparison.  This  crops  the  filtered 

%image  to  be  the  same  size  as  the  unfiltered  image. 

filtered_pixels  =  reshape(filtered_frame,  filtered_size(l)*filtered_size(2), 

filtered_size(3)); 

filtered  =  reshape(filtered_pixels,  filtered_size); 

fil_unfil_pixels  =  filtered_pixels  -  unfiltered_pixels;  %filtered  -  unfiltered  ->what  we 
want 

%This  takes  a  pixel-by-pixel  subtraction  (filtered- unfiltered)  to  show  what 
%the  filter  did  to  the  image. 

fil_unfil  =  reshape(fil_unfil_pixels,  unfiltered_size); 

%%  Now  we  need  to  convert  the  colors  to  ranges 
%Red 

phi_620  =  0.00689;  %(m-l)  temperature  and  salinity  corrected  absorption 
%coefficient  for  620  nm  at  27.6  C  in  .77  g/L  salt  water 
Red_Shift  =  log(filtered_pixels./unfiltered_pixels)/(-phi_620); 
min_Red  =  min(Red_Shift(isfinite(Red_Shift)));  max_Red  = 
max(Red_Shift(isfinite(Red_Shift))); 
for  i  =  l:length(Red_Shift(:,l)) 
ifRed_Shift(i,l)==  inf 
Red_Shift(i,l)  =  max_Red; 
elseif  Red_Shift(i,l)  ==  -inf 
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Red_Shift(i,l)  =  min_Red; 
end 
end 

Red_Matrix  =  reshape(Red_Shift,  filtered_size); 

Red_3D  =  Red_Matrix(:,:,l);  %red  pixels 

phi_510  =  0.00689;  %(m-l)  temperature  and  salinity  eorreeted  absorption 
%eoeffieient  for  510  nm  at  27.6  C  in  .77  g/L  salt  water 
Green_Shift  =  log(filtered_pixels./unfiltered_pixels)/(-phi_5 10); 
min_Green  =  min(Green_Shift(isfinite(Green_Shift))); 
max_Green  =  max(Green_Shift(isfinite(Green_Shift))); 
for  i  =  l:length(Green_Shift(:,2)) 
if  Green_Shift(i,2)  ==  inf 

Green_Shift(i,2)  =  max_Green; 
elseif  Green_Shift(i,2)  ==  -inf 
Green_Shift(i,2)  =  min_Green; 
end 
end 

Green_Matrix  =  reshape(Green_Shift,  filtered_size); 

Green_3D  =  Green_Matrix(:,:,2);  %green  pixels 
%  Blue 

phi_440  =  0.00331;  %(m-l)  temperature  and  salinity  eorreeted  absorption 
%eoeffioient  for  440  nm  at  27.6  C  in  .77  g/L  salt  water 
Blue_Shift  =  log(filtered_pixels  ./unfiltered_pixels)/(-phi_440) ; 
min_Blue  =  min(Blue_Shift(isfinite(Blue_Shift))); 
max_Blue  =  max(Blue_Shift(isfinite(Blue_Shift))); 
for  i  =  l:length(Blue_Shift(:,3)) 
if  Blue_Shift(i,3)  ==  inf 
Blue_Shift(i,3)  =  max_Blue; 
elseif  Blue_Shift(i,3)  ==  -inf 
Blue_Shift(i,3)  =  min_Blue; 
end 
end 

Blue_Matrix  =  reshape(Blue_Shift,  filtered_size); 

Blue_3D  =  Blue_Matrix(:,:,3);  %blue  pixels 
%  RGB  is  the  summation  of  Red  -i-  Green  -i-  Blue  values 
RGB_3D(:,:,1)  =  (Red_3D  -i-  Green_3D  -i-  Blue_3D); 

%%  Plot  results 

figure(l)  %original  unfiltered  image 

imshow(uint8(unfiltered)) ; 
figure(2)  %original  filtered  image 

imshow(uint8(filtered)); 

figure(3)  %This  shows  the  effeet  of  the  filter 

imshow(uint8(fil_unfil)); 

figure(4)  %3D  image  based  on  red  color  shift,  straight  on  view 
mframe  =  fliplr(Red_3D); 
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mesh(mframe( : , : ,  1 )  ,'LineWidth',  .3) 
shading  interp 

axis(  [0,length(Red_3D(  1 , : ))  ,0,length(Red_3D(: ,  1 ))  ,min_Red,max_Red] ) 

xlabel('x  (pixel)');  ylabel('y  (pixel)');  zlabel('z  (mm)') 

view(  180,90);  colorbar; 

shortname  =  strsplit(unf_img,'.'); 

shortname(l,2)=  {'Redl.jpg'}; 

newname  =  strjoin(shortname); 

saveas(gcf,newname,'jpg') 

figure(5)  %3D  image  based  on  red  color  shift,  rotated  view 
mframe  =  fliplr(Red_3D); 
mesh(mframe(: ,: ,  1  ),'LineWidth',  .3) 
shading  interp 

axis(  [0,length(Red_3D(  1 , : ))  ,0,length(Red_3D(: ,  1 ))  ,min_Red,max_Red] ) 

xlabel('x  (pixel)');  ylabel('y  (pixel)');  zlabel('z  (mm)') 

view(150,70);  colorbar; 

shortname(l,2)=  {'Red2.jpg'}; 

newname  =  strjoin(shortname); 

saveas(gcf,newname,'jpg') 

figure(6)  %3D  image  based  on  green  color  shift,  rotated  view 
mframe  =  fliplr(Green_3D); 
mesh(mframe( : , : ,  1 )  ,'LineWidth',  .3) 
shading  interp 

axis([0,length(Green_3D(l,:)),0,length(Green_3D(:,l)),min_Green,max_Green]) 

xlabel('x  (pixel)');  ylabel('y  (pixel)');  zlabel('z  (mm)') 

view(150,70);  colorbar; 

shortname(l,2)=  {'Green.jpg'}; 

newname  =  strjoin(shortname); 

saveas(gcf,newname,'jpg') 

figure(7)  %3D  image  based  on  blue  color  shift,  rotated  view 
mframe  =  fliplr(Blue_3D); 
mesh(mframe( :,:,!),  'LineWidth' , .  3 ) 
shading  interp 

axis([0,length(Blue_3D(l,:)),0,length(Blue_3D(:,l)),min_Blue,max_Blue]) 

xlabel('x  (pixel)');  ylabel('y  (pixel)');  zlabel('z  (mm)') 

view(150,70);  colorbar; 

shortname(l,2)=  {'Blue.jpg'}; 

newname  =  strjoin(shortname); 

saveas(gcf,newname,'jpg') 

figure(8)  %3D  image  based  on  red  +  green  +  blue  color  shift,  rotated  view 
mframe  =  fliplr(RGB_3D); 
mesh(mframe( :,:,!),  'LineWidth' , .  3 ) 
shading  interp 

axis(  [0,length(RGB_3D(  1 , : ))  ,0,length(RGB_3D( : ,  1 ))  ,min_Red,max_Blue] ) 
xlabel('x  (pixel)');  ylabel('y  (pixel)');  zlabel('z  (mm)') 
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view(150,70);  colorbar; 
shortname(l,2)=  {'RGB  .jpg'}; 
newname  =  strjoin(shortname); 
saveas(gcf,newname,'jpg') 
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